In ever increasing numbers, aircraft and satellites are being equipped with multispectral scanners used to gather earth resource data. To increase the accuracy of this data, the number of spectral channels included in the scanners has increased, from the 4-channel scanner on the ERTS-1 satellite, through the 13-channel scanner on Skylab, to the Bendix Corporation's 24-channel scanner in numerous aircraft. As the number of scanners in use increases and the number of channels per scanner increases, multispectral data is being generated in increasing quantities. As the amount of data generated increases, the problems associated with processing the data in order to derive meaningful information therefrom also increase.
In the prior art, two distinct but complementary approaches to the processing of multispectral scanner data have been followed. One such approach to such processing uses a variety of techniques to produce color maps of the ground area that are suitable for visual inspection and interpretation. One such technique is to use the intensity of one primary color (red, green, or blue) to represent the intensity of the reflected energy in one of three spectral channels. If these three-color images are superimposed (either photographically or with a color video system) then a full color map is obtained.
There are a number of limitations to the color maps produced in this manner. First of all, since one color is associated with one particular spectral channel of the data it is difficult to produce a map that uses data from more than three different spectral channels. On the other hand, multispectral scanners with up to 24 spectral channels have been built. Further, if one uses data from multiple-passes of the 4-channel ERTS multispectral scanner, then 8, 12 or 16 effective channels of data would not be uncommon.
A second approach to processing multispectral scanner data focuses on digital processing techniques and attempts to classify each ground resolution element in a given area. One such technique attempts to classify each ground resolution element based on some type of pattern recognition algorithm. Such algorithms will require ground truth information in order to train the classifier. Obtaining good ground truth information is one of the most difficult and costly parts of the classification process. Further, a parametric classification technique is generally used in which statistical parameters such as mean vectors and covariance matrices are estimated from ground truth information with the degree to which these estimates reflect the true statistics determining the operating performance of the classification method. However, the resultant difficulty in obtaining good estimates of these parameters when the number of channels increases indicates that the classification accuracy tends to deteriorate when more spectral data is used. Thus, the additional spectral information, which should increase the class discrimination, is not effectively used in parametric classification techniques.
To overcome the obvious disadvantages of the pattern recognition method of processing the data and include information from more than three channels, a number of digital processing techniques, including various clustering methods, such as correlation clustering, have been developed. The correlation clustering method is based on the assumption (implicit in all uses of multraspectral scanner data for classification purposes) that ground resolution elements with the same (or nearly the same) spectural signatures will belong to the same class. Starting with this assumption, those ground resolution elements with similar spectral signatures can be assigned parameters grouping them together into clusters which may then be processed to produce color maps by the use of display systems such as NASA's PMISDAS system at the Johnson Space Center in Houston.
A disadvantage of this type of approach is that the processing is done blindly. That is, the parameter selection and processing are done without being able to see what the resulting color map will look like. Having seen a map produced by initial selection of parameters, the classifier will most likely wish to change these parameters. This requires more off-line digital processing. It is not uncommon for such an iterative approach to take many days or weeks to produce an "ideal" color map of a given area.
It becomes apparent that, in order to reduce the processing time required to produce acceptable classification maps, an interactive system which will allow the operator to change the various parameters in real time is desirable. Such an interactive system would permit extraction of the maximum amount of information from the multispectral data in a minimum amount of time.
Further, such an interactive color display that is to operate in real time would preferably utilize a video display system. Assuming a 500 .times. 500 picture element (pixel) matrix video display that must be refreshed every 1/30 second, one sees that a 3.75 MHz data rate is required to refresh the video display. Such systems are available and in use today. However, existing systems will simply display a single image and do not process the multispectral data in any way.
What is desired is to be able to change the correlation parameters in "real-time" as observed by the operator. Suppose one tries to do this digitally. Assume that the calculation of a single parameter value requires only 5 basic operations, each taking 1 .mu.sec. For ERTS data this calculation must be done for each of the four channels and the results added (assume 1 .mu.sec per addition) to obtain the correlation parameter of a single pixel. Thus, it would take 23 .mu.sec to compute the correlation parameter from the spectral signature for each of the three colors. Therefore, each of the 250,000 pixels contained in a 500 .times. 500 pixel display would require 69 .mu.sec of computation which means that it would take over 17 sec. to change the video picture. This is obviously not the real-time operation that is desired.
The basic problem with digital computations is that there are too many pixels (250,000) and one can therefore afford to spend only about 1 .mu.sec to process each pixel if the entire calculation is to be completed in some fraction of a sec. This suggests that a substantial amount of parallel processing must be done if real time operation is to be achieved. Although digital computers with substantial parallel processing capabilities have been designed and built (such as the ILLIAC IV), they would not be suitable for use in the small type of dedicated system envisioned here.
Alternatively, optical processing offers the ultimate in parallel processing. In U.S. Pat. No. 3,984,671, entitled "Optical Process for Producing Classification Maps from Multispectral Data", a method is suggested by which holographic correlation techniques could be used to produce classification maps. In such a system all of the ground resolution elements are processed simultaneously at the speed of light. However, a real-time system would require a real time input transducer capable of changing coded data for all ground resolution elements at video rates as well as a real time medium for recording the holographic filters. While a number of such real-time devices and recording media are being developed in various laboratories, none at the present time possesses all of the properties that would be required for the type of interactive system disclosed in the present invention. Additionally, in order to make a color display it would be necessary to construct an elaborate system containing lasers of three different colors. Such an interactive real-time system using coherent optical processing is, at present, beyond the current state of the art.
The disadvantages of the prior art are overcome with the present invention, wherein a method and an apparatus are provided for producing color displays of multispectral imagery and which allow an operator to change parameters of the clusters (i.e., classification level changes) in an interactive, real-time manner.